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ABSTRACT 

An  iterative  scheme  is  given  for  solving  the  linear  complementarity 
problem  x  >  0,  Mx  +  q  2  0/  xT(Mx  +  q)  =  0.  At  each  iteration  the  main 
effort  is  to  find  a  point  x  satisfying  x  >  0,  Mx  +  q  >  0  and  a  (scalar) 
linear  inequality  which  is  easily  updated  iteratively.  It  is  shown  that  if 
M  is  a  P-matrix  or  positive  semidef inite ,  or  satisfies  a  fairly  weak 
positivity  assumption,  then  the  scheme  converges  to  a  solution.  The  scheme 
allows  many  possible  implementations  including  sparsity-preserving  methods  and 
direct  pivoting  methods.  An  iterative  linear  programming  method,  LCP-ILP,  is 
discussed  in  detail.  The  method  is  very  similar  to  the  Two- Phase  method  of 
linear  programming.  Numerical  results  are  presented  which  indicate  that 
LCP-ILP  is  more  effective  than  Lemke's  method  in  solving  general  linear 
complementarity  problems . 
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SIGNIFICANCE  AND  EXPLANATION 


The  linear  complementarity  problem  (LCP)  is  an  important  problem  in 
mathematical  programming  for  solving  linear  programming,  quadratic  programming 
and  bimatrix  games.  This  work  gives  a  new  scheme  for  solving  the  LCP.  Based 
on  the  scheme  different  algorithms  may  be  designed  for  finding  approximate 
solutions  of  very  large  scale  problems  or  exact  solutions  of  moderate  LCPs.  A 
specific  algorithm  for  the  latter  is  given,  together  with  some  numerical 
results.  These  indicate  that  the  method  is  more  effective  than  other  existing 


methods . 


The  responsibility  for  the  wording  and  views  expressed  in  this  descriptive 
summary  lies  with  MRC,  and  not  with  the  author  of  this  report. 
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AN  ITERATIVE  SCHEME  FOR  LINEAR  COMPLEMENTARITY  PROBLEMS 
Tzong-Huei  Shiau 

1.  INTRODUCTION 

we  propose  an  iterative  scheme  for  solving  the  linear  complementarity  problem  (LCP)  of 
finding  a  vector  x  in  the  n-dimenaional  Euclidean  space  tP  such  that 

(1.1)  x  >  0 ,  Mx  +■  q  >  0 ,  xT ( Mx  +  q )  -  0 

where  M  is  a  given  n-by-n  real  matrix  and  q  is  a  given  real  n-vector. 

At  each  iteration  of  the  scheme,  the  main  effort  is  to  solve  the  following  system  of 
2n  +  1  linear  inequalities,  (x  >  0  is  considered  n  inequalities). 

(1.2)  x  >  0,  Mx  +  q  >  0,  c£x  <  , 

where  ck  e  if1 ,  e  R  are  easily  determined  at  the  k-th  iteration.  Note  that  only  the 
last  inequality  of  (1.2)  is  changed  for  each  iteration.  Hence,  in  solving  (1.2),  we  can 
start  at  the  solution  of  the  previous  iteration  which  satisfies  all  except  possibly  the 
last  inequality.  This  changing  inequality  may  be  considered  as  a  cutting  plane  which  cuts 
off  some  protlon  of  the  feasible  region  {x|x  >0,  Mx  +  q  >  0}.  However,  the  scheme 
differs  from  general  cutting  plane  methods  in  that  the  number  of  cuts  remains  one  for  each 
iteration  while  other  cutting  plane  methods  accumulate  all  the  cuts  of  previous  iterations. 

It  is  shown  that  the  scheme  generates  a  "descending"  sequence  of  feasible  points  of 
the  quadratic  program, 

(1.3)  min  f(x)  s»xT(Mx+q)  subject  to  x  >  0,  Mx  +  q  >  0 

and  the  sequence  converges  to  a  Karush-Kuhn-Tucker  (KXT)  point  of  (1.3).  Note  that  any 
global  minimum  of  (1.3)  is  a  solution  of  (1.1)  provided  (1.1)  has  a  solution.  Therefore  in 
cases  where  any  KXT  point  of  (1.3)  is  a  global  minimum,  the  sequence  converges  to  a 
solution  of  (1.1),  e.g.  when  M  is  positive  semi-definite  or  when  M  is  a  P-matrix.  The 
convergence  to  a  solution  is  finite  if  in  solving  (1.2)  for  a  feasible  point  satisfying  the 
cut,  we  require  that  the  feasible  point  is  a  vertex  of  the  feasible  region. 

Sponsored  by  the  United  States  Army  under  Contract  No.  DAAG29-80-C-004 1 .  This  material  is 
based  on  work  supported  by  the  National  Science  Foundation  under  Grant  No.  MCS-8200632. 
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The  scheme  is  very  flexible  that  it  allows  many  possible  implementations.  Among  them 
the  iterative  linear  programming  method  (LCP-ILP)  is  discussed  in  detail  and  some  numerical 
results  are  presented  which  indicate  that  for  general  linear  complementarity  problems 
LCP-ILP  is  more  efficient  and  solves  more  porblems  than  Lemke's  method  (5).  In  some  other 
paper  we  shall  discuss  other  implementations  which  preserve  sparsity  and  are  more  suitable 
for  very  large  scale  problems  with  sparse  M.  Unlike  most  of  other  sparsity-preserving 
methods  !e.g.  see  3,  6,  8,  9],  the  algorithms  can  solve  nonsymmetric  LCP's  when  M  is  only 
positive  semidef inite . 

We  define  some  notations  and  terms  used  in  the  paper.  An  x  e  Rn  is  feasible  if 
x  >  0,  Mx  +  q  >  0.  The  set  of  all  feasible  points  is  called  the  feasible  region.  The  LCP 

(1.1)  is  feasible  if  the  feasible  region  is  nonempty.  Vf(x)  :=  (M  +  MT)x  +  q  is  the 
gradient  of  f(x)  :=  xT(Mx  +  q)  as  defined  in  (1.3).  x  is  a  solution  if  x  is  feasible 
and  f(x)  =0.  x  is  a  vertex  if  it  is  an  extreme  point  of  the  feasible  region.  By 
Condition  (A),  we  mean 

Condition  (A)  There  exists  a  solution  x  of  (1.1)  such  that 
(x  -  x)tm(x  -  x)  >  0  for  all  feasible  x. 

If  (1.1)  is  feasible,  then  Condition  (A)  holds  when  M  is  positive  semidefinite.  In  fact. 

Condition  (A)  is  equivalent  to  that  f,  restricted  to  the  feasible  region,  is  convex  at 

some  solution  x.  From  now  on  we  assume  (1.1)  is  feasible. 

1  k  k+ 1 

Superscripts  are  used  to  denote  different  vectors,  e.g.  x  ,x*  and  x  ,  but  the 

T  T 

superscript  T  denotes  transpose,  e.g.  x  ,  M  . 

2.  THE  BASIC  IDEA 

The  followina  lemma  shows  that  given  a  feasible  point  which  is  not  a  solution ,  we  have 

a  cutting  plane  separating  the  point  from  a  solution  under  a  very  weak  convexity 

assumpt ion , 

2.1  Lemma  ■  let  xk  be  feasible  and  f(xk)  >  0.  Then  the  linear  inequality 

(2.1)  f(xk)  +  Vf(xk)(x  -  xk)  <  0 

which  is  violated  by  xk  is  satisfied  by  any  solution  z  satisfying 
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(2.2) 

(xk  -  z)TM(xlt  -  z)  >  0  . 

Proof . 

Obviously  xk  violates 

(2.1)  since  f(xk)  >  0.  Let  z  be  any  solution  satisfying 

(2.2). 

Then  the  equation 

(2.3) 

f(xk)  + 

7f(xk)T(z  -  X*)  -  -(xk  -  z)TM(xk  -  z) 

can  be  shown  either  by  algebraic  manipulation  or  by  equating  the  zeroth,  first  and  second 
order  derivatives  of  both  sides  with  respect  to  xk  at  z  (note  that  f(z)  ■  0).  It 
follows  by  (2.3)  and  (2.2)  that 

f(xk)  +  Vf(xk)T(z  -  xk)  <  0 

i.e.  z  satisfies  (2.1).  □ 

Lemma  2.1  leads  naturally  to  the  following  algorithm 

2.2  The  Cutting  Plane  Method 

(0)  Initialization.  Let  S  be  the  set  of  the  2n  constraints  x  £  0,  Mx  +  q  £  0. 
Let  x°  be  any  feasible  point. 

(1)  Iteration.  Given  xk,  k  >  0,  add  the  cut  (2.1)  to  S,  find  xk+1  satisfying 
all  constraints  in  S.  □ 

Note  that  the  number  of  constraints  in  S  increases  by  one  for  each  iteration.  It  is 
simple  to  establish  the  following: 

2.3  Convergence  Theorem  of  the  Cutting  Plane  Method.  Let  {x  }  be  generated  by  Method 

2.2.  Then  any  accumulation  point  z  of  {x  }  is  a  solution. 

k.  k3+1 

Proof .  Let  {x  3}  be  a  subsequence  converging  to  z.  Since  x  3  satisfies  all 

N 

previous  cuts,  in  particular  it  satisfies  the  cut  defined  by  x  .  Hence 

f(x  +  Vf(x  (x  ^  -  X  <  0 

Taking  the  limit  as  j  ♦  •»,  we  have  f(z)  <  0.  Hence  z  is  a  solution  since  it  is 
clearly  feasible.  □ 

It  should  be  pointed  out  that  Theorem  2.3  did  not  say  that  the  set  of  points 
satisfying  all  constraints  in  S  is  always  nonempty  as  more  and  more  cuts  are  added  into 
S.  However,  it  is  indeed  the  case  when  Condition  (A)  holds  since  x  satisfies  all  cuts  by 
Lemma  2.1. 
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Like  other  cutting  plane  algorithms,  the  method  suffers  from  the  following 
difficulties.  As  the  number  of  constraints  increases,  x  becomes  more  and  more 
difficult  to  compute,  and  usually  the  convergence  becomes  slower  and  slower.  Forturnately , 
we  have  found  a  simple  remedy.  By  doing  an  additional  line  search  in  each  iteration,  none 
of  previous  cuts  need  be  kept. 

3.  THE  ITERATIVE  SCHEME 

3 . 1  The  Basic  Scheme 

(0)  Initialization.  Let  x°  be  any  feasible  point. 

(1)  Iteration.  Given  xk,  stop  if  fix'*)  =  0.  Otherwise  let  yk  be  any  feasible 

U  le  V 

point  satisfying  the  cut  (2.1).  Line-search  along  the  direction  p  y-  x 
using  the  minimization  stepsize  xk+^  :=  xk  +  tkpk  where 

(3.1)  tk  !=  arg  minimum  fix*  +  tpk)  •  □ 

0<t<1 

k  )c  v 

3.2  Theorem.  Let  (x  ,y  }  be  generated  by  the  Iterative  Scheme  3.1.  If  (y  }  is 

bounded,  then 

f(xk)  +  0  as  k  ♦  -  . 

Proof.  See  Appendix.  □ 

It  should  be  noted  again  that  without  any  convexity  assumption  such  as  Condition  (A), 
there  may  be  no  feasible  point  satisfying  (2.1)  in  the  k-th  iteration.  We  call  (2.1)  an 
overcut  if  it  is  the  case.  The  following  lemma  shows  that  we  can  still  get  a  descent 
direction  pk  by  a  weaker  cut  parallel  to  the  overcut. 

3.3  Lemma.  Let  xk  be  feasible.  Then 

(3.2)  Vf(xk)T(x  -  xk)  >  0  for  all  feasible  x 

iff 

(3.3)  xk  is  a  KXT  point  of  (1.3)  . 

Therefore,  if  xk  is  not  a  KXT  point  of  (1.3),  then  for  some  0^  >  0  sufficiently  small, 
there  is  a  feasible  point  x  satisfying 


(3.4) 


Vf(x*)  (x  -  x*)  +  8,  f (x  )  <  0  . 

k  « 

Proof .  (3.2)  holds  iff  xk  Is  an  optimal  solution  of  the  linear  program 

(3.5)  minimize  Vf(x  )x  subject  to  x  >  0,  Mx  +  q  >  0 

Since  the  optimality  condition  of  (3.5)  for  xk  is  precisely  the  KKT  condition  of  (1.3) 
for  xk,  (3.2)  and  (3.3)  are  equivalent.  Hence  Vf(xk)(x  -  xk)  <  0  for  some  feasible 
x  if  x*  is  not  a  KXT  point  of  (1.3).  Therefore  (3.4)  holds  for  8^  sufficiently  small.  □ 
The  cut  (3.4)  is  a  generalization  of  (2.1)  that  can  be  used  when  (2.1)  is  an 
overcut.  This  leads  the  general  scheme. 

3.4  The  General  Iterative  Scheme 

(0)  Initialization.  Let  x°  be  any  feasible  point 

(1)  Iteration.  Given  xk,  stop  if  xk  is  a  KKT  point  of  (1.3).  Otherwise, 
let  y**  be  a  feasible  point  satisfying 

(3.6)  Vf(xk)T(yk  -  xk)  +  0kf(xk>  <  0 

where  8^  is  a  positive  number.  Let  xk+1  s-  xk  +  tkpk  where  pk  »»  yk  -  xk  and  tk  is 
the  minimization  stepsize  defined  in  (3.1).  □ 

Ideally  8fc  should  be  1  if  admissible.  If  “  1  leads  to  an  overcut,  we  can  keep 
on  reducing  it  by  half  until  (3.6)  holds  for  some  yk.  That  is,  we  choose  ®k  “  1  or 
choose  8 ^  large  enough  that  28^  will  lead  to  an  overcut.  The  following  theorem  shows 
that  we  will  get  a  solution  or  a  KKT  point  of  (1.3). 

3.5  Theorem.  Let  {xk,yk}  be  generated  by  the  General  Iterative  Scheme  3.4.  Assume  xk 
is  not  a  KKT  point  of  (1.3)  for  k  >  0.  Assume  {y  }  is  bounded.  Then 

(1)  {xk}  is  bounded,  {f(xk)}  is  decreasing  and 

lim  Vf(xk)Tpk  -  0 

(ii)  If  there  exists  6  >  0  such  that  for  infinitely  many  k,  8^  >  5 ,  then 

lim  f(xk)  -  0 
k+« 

(iii)  If  there  exists  5  >  0  such  that  for  all  k,  either  8^  >  6  or 

(3.7)  Vf (xk)T(x  -  xk>  26,  f(xk>  <  0 

k  ■ 

is  an  overcut ,  then  any  accumulation  point  of  {x  }  is  a  KKT  point  of  (1.3). 
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Proof .  See  Appendix.  □ 


L- 

K' 

K 

► 


<*. 

f. 


% 


Theorem  3.5  shows  that  without  any  convexity  assumption,  the  scheme  will  "converge"  to 
a  KKT  point  of  (t.3).  In  cases  that  we  can  keep  8^  from  converging  to  zero,  e.g. 

8-1  if  Condition  (A)  holds,  then  a  solution  is  obtained.  The  scheme  also  converges  to 
a  solution  in  cases  in  which  any  KKT  point  is  a  solution  as  shown  below. 

3.6  Corollary.  If  (i)  M  is  positive  semi-definite,  or  (ii)  M  is  a  P-matrix,  or  (iii) 

M  is  quasi-diagonally  dominant,  i.e.  there  exists  d  =  (d.)  ,d2 ,  . . .  ,dn)T  S  R11,  >  0, 

such  that 

(3.8)  Miidi  >  l  iMijjdj,  i  =  1,2, ...,n  , 

j*i 

then  any  accumulation  point  of  {x  }  is  a  solution. 

Proof ■  For  <i),  f(x)  is  convex.  Hence  any  KKT  point  of  (1.3)  is  a  global  minimum  of 
(1.3)  which  is  a  solution  of  (1.1).  For  (ii),  any  KKT  point  is  a  solution  [1],  which  is 
also  true  for  (iii)  (10,  Thm.  5.5.1].  □ 

4.  A  SIMPLEX-BASED  IMPLEMENTATION 

The  main  effort  in  each  iteration  is  to  find  a  feasible  y^  satisfying  the  cut.  The 
quality  of  y*  will  affect  the  speed  of  convergence  of  the  method.  In  this  section  we 
present  a  simplex-based  method  to  find  yk  which  is  a  vertex  of  the  feasible  region.  This 
is  desirable  because  at  least  one  of  the  vertices  is  a  solution  if  (1.1)  is  solvable  [7], 
4.1  The  Simplex-Based  Method  (LCP-ILP) 

(0)  Initialization.  Use  phase  I  of  the  two-phase  simplex  method  to  find  a  basic 
feasible  solution  (BFS)  and  the  corresponding  simplex  tableau  for  the  linear 
program 

(4.1)  min  0*x  subject  to  Mx  +  q  >  0,  x  >  0  . 

Let  x1  =  y°  =  the  BFS.  (This  step  is  not  needed  if  a  BFS  is  available). 

(1)  Iteration .  Given  xk  feasible  and  a  vertex  y^-1  and  its  tableau,  recompute  the 

k 

reduced  cost  vector  using  Vf(x  )  as  the  new  cost  vector,  then  continue  the 
primal  pivot-steps  until  either  (i)  the  current  BFS  satisfies  the  cut  (2.1),  or 


5-. 

f. 

:3 
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(ii)  the  BFS  is  optimal  for  the  cost  vector  whichever  occurs  first* 


Let  yk  be  the  BFS,  compute  xk+1  by  the  line  search  described  before.  □ 
Roughly  speaking,  the  method  is  very  close  to  the  Two-Phase  Simplex  method  for  solving  the 
linear  program 

(4.2)  min  cTx  subject  to  Mx+q>0,  x  >  0  . 

The  only  difference  is  that  c  «  Vf(x  )  is  updated  whenever  the  current  E’S  yK  satisfies 
the  cut  (2.1),  i.e.  the  objective 

(4.3)  cTyk  -  7f(xk)Tyk  <  7f(xk)Txk  -  f(xk)  -  xkMTx3 

which  is  very  easy  to  test,  or  when  yk  is  optimal  for  c  »  7f(xk).  In  the  latter  case, 

v 

y  satisfies  the  generalized  cut  (3.4)  with 


(4.4) 


I 


k.T,  k  k. 

e  !=  .  7fi*  >  <y  >  >  o 

k  f(xk) 


where  8^  »  0  iff  xk  is  optimal  for  (4.2)  (with  c  •  Vf(xk)).  If  6^  ■>  0  then  xk  is 
a  KXT  point  of  (1.3),  otherwise  xk+1  is  determined  by  a  simple  line-search  and  c  and 

(defined  in  (4.3))  are  easily  updated.  Note  that  28fc  leads  to  an  overcut  since  8k 
is  the  largest  number  which  does  not. 

If  the  reader  is  careful  enough,  she/he  may  ask  what  if  the  simplex  method  leads  to  a 
unbounded  ray.  This  is  the  last  problem  we  need  to  solve  before  the  convergence  results  in 
Section  3  are  applicable.  The  answer  is  very  simples  it  never  happens. 

4.2  Theorem.  Let  xk  be  feasible,  then  the  LP 

min  Vf(xk)Ty  subject  to  My  +  q  >  0,  y  >  0 


is  bounded. 

Proof .  Let  d  be  any  direction  of  the  feasible  region,  i.e.  there  is  a  feasible  y  such 
that  y  +  Xd  is  feasible  for  all  X  >  0.  Then  d  >  0  and  Md  >  0.  Hence 

Vf (xk)Td  »  ( Mxk  +  q)Td  +  (xk)T(Md)  >  0  . 

Therefore  no  direction  is  a  descent  direction  and  LP  is  bounded.  □ 

The  following  theorem  shows  that  the  convergence  of  Simplex-Based  Method  (LCP-ILP)  is 
finite  if  it  is  not  trapped  by  a  non-solution  KKT  point.  Hence  if  0.  is  away  from  zero, 


e.g.  Condition  (A)  holds,  or  one  of  the  conditions  of  Corollary  3.5  is  true,  then 
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xk+1  =  yk  is  a  solution 

for 

some  k . 

4.3  Theorem.  Let  U  :=* 

(z  1  z 

is  a  vertex,  but 

not  a 

solution} 

be  the  finite  set  of  non- 

m 

solution  vertices.  Let 

conv(U)  be  its  convex 

hull. 

let  6  :» 

inf {f ( x ) |x 

e  conv ( u ) } . 

u 

jum 

Then  LCP-ILP  either 

m 

(i)  generates  a  sequence 

{x  }  such  that 

f(xk) 

O 

A 

«o 

A 

for  all  k. 

or  (ii) 

h 

terminates  at  a  solution 

xk 

for  some  k  >  1 , 

in  which  case 

yk-1  is  a 

vertex  solution 

Proof .  Since  U  is  finite,  conv(O)  is  compact.  Hence  there  is  a  vector,  say  x,  in 
Conv(U)  such  that 

f(x)  =  6  =  inf{f(z)|z  e  conv(U)}  . 


Let  x  *  EX^z*,  zk  e  u»  >  0,  =*  1.  Picking  any  zK,  say  s’,  there  is  an 


-X  _ k  a  1 1  \  V  A  r  \  —t  4  nJ  nl.4  nj.  BMtt  .k  «.it  .  ^ 

2  f 

index  i,  1  <  i  <  n,  such  that  z^  >  0  and  (Mz1  +  q)|  >  0  since  z1  is  not 


complementary .  Hence 


and 


^y2i  *  ^izi  >  0  • 


(Mx  +  gjj  *  rXfc(Mzk  +  q)i  >  +  q)±  >  0 


So 


6  “  f(x)  =  \  x j (Mx  +  q)^  >  Xj(Mx  +  q)^  >  0  . 

Suppose  yk  e  U  for  0  <  k  4  N,  then  it  is  easy  to  see,  by  induction,  that 
xN+1  e  conv(U).  So  f(x"+1)  >  6  >  0.  This  shows  that  either  one  of  y*  is  a  vertex 
solution  or  f(xk)  >  5  for  any  xk  generated.  In  the  formal  case,  xk+1  »  y^  is  the 
solution  obtained  by  the  line-search.  □ 

4.4  Corollary.  LCP-ILP  finds  a  vertex  solution  in  a  finite  number  of  iterations  if  one  of 
the  followings  is  true. 

(i)  Condition  (A)  holds. 

(ii)  M  is  positive  semi-definite. 

(iii)  M  is  a  P-matrix. 

(iv)  M  is  quasi-diagonally  dominant. 


-8- 


Proof .  For  (i),  the  cut  (2.1)  is  not  an  overcut.  Therefore  6^  >  1  at  each  iteration. 

It  follows  by  Theorem  3.5  (ii)  that  case  (i)  of  Theorem  4.3  is  impossible. 

For  (ii),  (iii)  and  (iv),  any  KXT  point  is  a  solution  as  is  pointed  out  in  the  proof 
of  Corollary  3.6.  Suppose  LCP-1LP  does  not  find  a  solution  in  a  finite  number  of 

l- 

iterations,  we  shall  show  a  contradiction.  By  Theorem  3.5,  since  x  ,  k  >  0,  are  not  KKT 

points,  LCP-ILP  generates  a  bounded  sequence  {x  }  having  an  accumulation  point,  say  x, 

which  is  a  KKT  point.  Hence  f(x)  =  0  since  a  KKT  point  is  a  solution.  On  the  other 

hand,  ftx*')  >  6  >  0  for  all  k  by  Theorem  4.3.  Hence  f(x)  >  lim  inf  fix1')  >  6  >  0.  □ 

k-x» 

4.5  Numerical  Results.  A  computer  program  implementing  LCP-ILP  was  written  and  tested  on 
UWVAX,  a  VAX  11/780  computer  running  UNIX  operating  system.  All  floating  point 
computations  are  in  dobule  precision  which  provides  about  16  figure  decimal  accuracy.  Six 
problem  sets  were  randomly  generated;  each  has  20  individual  problems.  The  dimension  n 
is  fixed  in  each  set  and  varies  among  sets.  For  the  purpose  of  comparison  Lemke's 
Algorithm  was  also  tested  for  these  problems.  Recall  that  LCP-ILP  may  converge  to  a  KKT 
point  which  is  not  a  solution,  while  Lemke's  Algorithm  may  terminate  at  a  ray  and  fail  to 

give  a  solution.  In  fact,  since  the  general  LCP  is  NP-complete  [2],  there  is  no  known 

algorithm  that  is  guaranteed  to  solve  any  LCP  without  essentially  enumerating  an 
exponential  number  of  possible  cases  which  is  impossible  to  do  even  for  moderate  n. 

The  results  are  shown  in  Table  1.  We  make  the  following  remarks. 

(i)  The  iteration  number  and  the  pivot  number  are  the  same  for  Lemke’s  method,  but 
different  for  LCP-ILP.  For  the  latter,  iteration  number  is  the  number  of 
updatings  of  the  cost  vector  while  the  pivot  number  is  the  total  number  of 
pivots  of  all  iterations.  Since  updating  the  cost  vector  is  easy  and  the  number 
of  updating  is  significantly  smaller  than  the  pivot  number,  we  feel  the  pivot 
number  is  a  good  measure  of  computation  time.  Also  the  comparison  between  pivot 
numbers  of  these  two  algorithms  is  a  good  estimate  of  the  relative  speed  since 
each  pivot  takes  roughly  the  same  time  in  either  algorithm. 

(ii)  The  phase  I  of  I/1P-ILP  starts  at  the  origin  to  find  the  initial  BFS .  The  number 
of  pivots  given  in  Table  1  includes  the  pivot  steps  taken  in  phase  I. 


(iii)  Although  Lemke's  Algorithm  is  very  effeicient  for  M  being  a  P-matrix  or 
positive  semi-definite,  it  rarely  gives  a  solution  for  a  general  M  when 
n  >  30.  Moreover,  it  becomes  very  slow  no  matter  whether  it  gives  a  solution  or 
not.  Hence  the  so  called  almost  complementary  path  has  a  very  long  average 
length. 

(iv)  The  number  of  problems  solved  in  a  set  decreases  rapidly  as  n  increases.  This 
is  not  surprising  since  the  problem  is  NP-complete  [2]. 

(v)  The  ratio  of  the  maximum  number  to  the  minimum  number  of  pivots  of  a  set,  given 
in  the  bottom  row  of  each  column,  remains  small  for  LCP-ILP  but  varies 
considerably  for  Lemke's  method.  Hence  the  almost  complementary  path  can  be 
very  short  as  well  as  very  long.  It  also  indicates  that  LCP-ILP  is  more  robust, 

(vi)  since  LCP-ILP  can  start  at  any  BPS,  it  will  be  much  faster  if  we  have  a  BFS  to 

begin  with  or  each  problem  set  is  a  parametric  LCP  in  which  case  it  can  start  at 
the  solution  basis  of  the  previous  problem.  It  should  be  noted,  however,  that 
some  generalizations  of  Lemke's  method  can  start  at  an  arbitrary  point  under 
certain  assumptions  (4,  11], 
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APPENDIX 

Theorem  3.2  is  a  special  case  of  Theorem  3.5  with  8^  =  1.  (Note  that  xk  is  not  a 
KKT  point  in  Theorem  3.2.  Otherwise  by  Lemma  3.3  there  would  be  no  point  satisfying  the 
cut  (2.1)  and  xk+1  could  not  be  generated).  Hence  we  shall  only  give  the  proof  of 
Theorem  3.5. 

For  simplicity,  we  introduce  the  following  notation. 

( A.  1 ) 

(A. 2)  a  f(xk),  B  Vf(xk)Tpk,  y  :=  <pK)AMpK 

It  is  easy  to  see  that  $(t)  is  a  quadratic  function  and  that  $(0)  =a,  ♦'(0)  =6, 

$ '  ( o )  =  3y .  Hence 


$(t)  :  =  f (xk  +  tpk) ,  t  e  R 


„k>TM„k 


(A. 3) 


4>  ( t:  >  S  a  +  Bt  +  ft 


:§ 

w\v 

k'.N 

£ 

Lemma 

( 

9 

Proof . 

\/ 

since 

w, 

«  m  * 

Case  1 

(i)  t. 


=  0 


Zl 

2y' 


if  0  <  t  <  1 
otherwi se 


where  tk  is  the  stepsize  defined  by  (3.1). 


2  k 


Proof .  By  assumption  xk  is  not  a  KKT  point,  hence  a  >  0.  By  (3.6),  $  <  -  0a  <  0 


Hence  y  >  -  —  B  >  0,  $(t)  is  strictly  convex  with  the  global  minimum  t.  Hence 


tk  =  t  and 


f(xk+1)  -  f(xk)  -  tk(8  +  tkf)  -  tk(B  +  f)  “  J  tkB 


proving  (ii). 
Case  2.  t  >  1. 


So  0  <  )  <  ■  y  8  and  <^ ( t )  is  again  strictly  convex.  Since  t  >  1,  $(t)  is 


strictly  decreasing  on  [0,1].  So  tk  “  1  and 


f(xk+1)  -  f(xk)  »B+f<B-j6=|tk8 


proving  (ii). 


/  / 


1  and 


Case  3 .  t  <  0 . 

■  m 

So  Y  <  0  and  $(t)  is  strictly  decreasing  on  [0,1].  Hence  tk 
f(xk+1)  -  f(xk)  -  8  +  Y  <  6  <  j  tk6  • 

The  proof  is  complete.  O 
Proof  of  Theorem  3.5.  Since  {y  }  is  bounded,  there  is  a  positive  number  V,  that 

|x°|  <  V,  | j  <  V  for  all  k  >  0  . 

It  is  easy  to  see,  by  induction,  that  xk+1  is  a  convex  combination  of  x°,y°,...,yk  < 

hence 

lxk|  <  V  for  all  k  >  0  . 

Hence  |pk|  =  |yk  -  xk|  <  ly11)  +  |xk|  <  2V  and 

(A. 4)  I  Y  |  *  l(pk)TMpk|  <  IMI  •  |  pk  |  2  <  II  Ml  »4V2  »!  W 

for  all  k  >  0.  Now  applying  Lemma  A.1  (i)  and  (ii),  we  have 

(A. 5)  f<xk)  -  f(xk+1)  >  -ytkS 


>  min{-  I  0,  -  f  -2jfr> 

“  min{-  j  8, 

1  S2 

>  min{-  j  8,  -fij]  (by  (A. 4)) 


Summing  up  (A. 5)  for  k  ■  0,...,N,  we  have 


(A. 6)  f ( x° )  -  f(xN+1)  >  l  6 

k=0  k 


The  positive  series  \  4,  converges  since  the  left  hand  side  of  (A. 6)  is  bounded  for 

k-0 

all  N.  Hence  lim  6.  »  0.  It  follows  by  the  definition  of  5.  in  (A. 5)  that 
k+®  *  * 

lim  -8  -  lim  -Vf(xk)Tpk  -  0 


So  by  (3.6), 

(A. 7)  lim  sup  9kf<xk)  <  lim  -Vf(xk)Tpk  -  0 

k-*»  k+“ 

On  the  other  hand,  since  {f(x  )}  is  strictly  decreasing  by  (A. 5),  if  8^  >  6  infinitely 
often  we  have 

lim  sup  6  f(xk)  >  (lim  sup  8  )*(lim  f(xk))  >  5*lim  f(xk) 

f+«  k+“ 

Toaether  with  (A. 7),  we  have  lim  f(xk)  =  0  proving  (ii). 

k+“ 

For  (iii),  because  of  (ii)  we  can  assume  that  (3.7)  is  an  overcut  for  all  k  >  K  for 
some  K.  Fix  any  feasible  y.  For  k  >  X,  since  y  cannot  satisfy  (3.7),  we  have 

Vf(xk)T(y  -  xk)  >  -28kf(xk) 

SO 

lim  inf  Vf(xk)T(y  -  xk)  >  lim  inf  -28,  f(xk) 
k-  =  k-  k 

-  -2  lim  sup  8  f(xk)  >  0  (by  (A. 7)) 

k-n» 

-  ki  -  t  - 

Let  x  be  any  accumulation  point  of  (x  } ,  then  Vf(x)  (y  -  x)  is  an  accumulation  point 

of  {Vf(xk)T(y  -  xk)},  hence 

(A. 8)  Vf (x)T(y  -  x)  >  lim  inf  Vf(xk)T(y  -  xk)  >  0 

k+« 

Since  (A. 8)  holds  for  any  feasible  y,  x  is  a  KXT  point  of  (1.3)  by  Lemma  3.3.  D 
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